Orthogonal Low Rank Tensor Approximation: Alternating Least Squares Method and Its Global Convergence

With the notable exceptions of two cases — that tensors of order 2, namely, matrices, always have best approximations of arbitrary low ranks and that tensors of any order always have the best rank-one approximation, it is known that high-order tensors may fail to have best low rank approximations. When the condition of orthogonality is imposed, even under the modest assumption that only one set of components in the decomposed rank-one tensors is required to be mutually perpendicular, the situation is changed completely — orthogonal low rank approximations always exist. The purpose of this paper is to discuss the best low rank approximation subject to orthogonality. The conventional high-order power method is modified to address the orthogonality via the polar decomposition. Algebraic geometry technique is employed to show that for almost all tensors the orthogonal alternating least squares method converges globally.